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TEST ESTIMATION 



Shyam Sunder* 



Abstract: 



Introduction: 

We have executed many projects Large Projects, Small Projects etc. Sometimes we miss our 
testing deadlines because there is no defined criterion that is used to build our execution test 
plan. To help avoid such missing our deadlines we have prepared these Test Estimation 
guidelines. 

Audience: 

• Test Management Professionals — Managers, Architects, Test Leads, Software Test 
Engineers 

• Quality / SEPG Management Professionals — Managers, QA Leads, Software QA Engineers 

• Project Management Professionals — Managers, Leads, Software Engineers 

Area of Application: 
Testing Projects 

Benefits: 

Why good estimates are important? 

• Testing is often blamed for late delivery 

• Testing time is "squeezed" 

• It promotes early risk assessment 



* Test Program Manager 
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Issues and Challenges: 

• Major Issues : Estimation method / Process, Environment & Tools, Testing to be performed, 
Complexity of the application under tests 

• Minor Issues : Test Resource, Other factors 



Issues While Estimating - Process 
Requirement Stability, Change Requests, Finalize types of testing ,Follow the test procession- 
availability of test cases and test data,Timely reviews of the test cases and other artifacts, 
Coordination with various teams/modules/interfaces, Scope changes 

Issues While Estimating - Environment & Tools 
Separate Environment, Environment not similar to the deployment environment, Downtime of 
the environment available during testing, Availability of test management tools, Availability of 
test automation tools. 

Issues While Estimating - Testing Resources 
Management commitment towards completion and following the test life cycle, Common and 
realistic expectation toward the testing goal from all the stakeholder in project, Availability of 
key resources, Application knowledge among the test team, Connect / Attitude between the 
development and testing team, Correct resolution on the defect fixes, Clear Communication 

Issues While Estimating - Others factors 
Complexity of the application under tests, incorrect assumptions during estimation, Ownership 
of testing, Independent \ Development Vendor\ Customer 

Timeline assigned for testing, Development timeline and release of code for testing, Availability 
of correct test data during test execution, Geographical location of testing team involved, Not 
doing periodic re-estimation 
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We have executed many projects Large Projects, Small Projects etc. Sometimes we miss our 
testing deadlines because there is no defined criterion that is used to build our execution test 
plan. To help avoid such missing our deadlines we have prepared these Test Estimation 
guidelines. In this paper I present the various Test estimation techniques which will help us in 
proper execution of the Testing projects. This is a paper submitted in the Test Management 
Stream. 



2.0 Content 



Test Estimation is a prediction based on probabilistic assignments and is a continuous process, 
which should be followed and used through out the project life cycle. Effective software 
estimation helps track and control cost/effort overruns. Estimations cover following broad areas 

Estimate size 

Estimate cost & effort 

Determine the schedule 

Assess risks 



Now this brings us to a basic question that how can we do Test Estimations? 
Test Estimations can be done as shown in the below figure. 
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Test Specifications 




Test team 




Test 
Environment 
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Quality 
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/ Complexity 



Product Exp. 



Test Capability 



Test Platform 



Test Tools 




Test Estimates 



Test 
Estimation Model 




There are various types of Test Estimations as shown below: 




The specifications 
of a given test suite 



23.5 
man-hours 

Estimated test 
execution effort 



Manual Test Exec i if inn i Effmt Estimation 




The specifications 
of a given test suite 



> 23.5 
man-hours 

Estimated test 
automation effort 



Test Airtom.it ion Effort Estitn.it ion 



Quality Criteria 
Th e s pe c it! c atio ns UP 



Teste sorted to maximize 
of a given test suite coverage with minimal effort 

^ %, 

Maximum Effort 
(optional) 

Teste to not 
execute 

Test Coverage x Execution Effort Analysis 




Teste to execute 
manually 

Cost-benefit Analysis to Prioritize Tests for Automation 
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Software Test Estimation - Overview 



C 



Budget 
Baseline Data 
Release Road map 
Type of testing 
Application size 




Estimate Test Size 



Estimate Cost ft Effort 



Determine Schedule 



Track & Compare 
Actual / Estimated 



Fine tune the 
estimation process 



Re- Estimate 



There are various issues which creep up while estimating 
Major Issues 

Estimation method / Process 

Environment & Tools 

Testing to be performed 

Complexity of the application under tests 



Minor Issues 

Test Resource 
Other factors 



Issues While Estimating - Environment & Tools 
Separate Environment 

Environment not similar to the deployment environment. 
Downtime of the environment available during testing 
Availability of test management tools 
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Availability of test automation tools 



Issues While Estimating - Testing Resources 

Management commitment towards completion and following the test life cycle 
Common and realistic expectation toward the testing goal from all the stakeholder in 

project 

Availability of key resources 

Application knowledge among the test team 

Connect / Attitude between the development and testing team 

Correct resolution on the defect fixes 

Clear Communication 



From the above we have got a basic idea about the facts related to Test estimations. 

There are various software test estimation techniques: 

a) Simple Medium Complex(SMC) Method 

b) Top Down Method 

c) Bottom Up Method 

d) Test Point Analysis(TPA) 

Now in the following sections I'd be explaining the above mentioned estimation techniques in 
detail. 



a) SMC Method 

This model will consider the test functions / test conditions and their 
Complexities (Simple, Medium, Complex) as the basis for estimation and 
The effort involved for the following test activities can be estimated using 
This model. Following test activities could be covered 
• Test Initiation 

Test Planning & Design 
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Test Execution 

Test Closure activities 

Test Initiation: 

Effort estimate for the following activities can be done using SMC model under Initiation Phase: 
Knowledge Transfer 
Application Familiarity 
Requirements Analysis 
Functional Decomposition 

Test Planning and Design: 

Effort estimate for the following activities can be done using SMC model under Test Planning & 
Design Phase: 

Test Plan 

Preparation of Scenarios, Test Cases, Test Data. 

Test Case, Test Data Reviews 

Preparation of Execution Plan 

Test Ware Re- work & Reviews 

Prepare and Review of Zero-day checklist 

Test Execution: 

Effort estimate for the following activities can be done using SMC model under Test Execution 



Phase: 



Verify zero day check list 
Creation of test bed 
Test Execution 
Review of Incident logs 
Update the Incident report 



Test Closure: 
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Effort estimate for the following activities can be done using SMC model under test 
closure Phase: 

Closure Metrics Preparation 
Closure meeting 

Archive project data (Project Closure Activities) 
Test / Project Management 



I'm attaching a Sample SMC Sheet which I had used for one of my test projects. 



C:\Sample SMC.xls 



a) Top Down Method 

In this method, the Overall effort estimate for the project is determined first in FP or Line of 
code method. The estimation procedure is as follows: 

• Get the total size in FP 

• Define the lower level project test component. 

• Based on experience and productivity data from previous projects, 
obtain the effort estimate 

• Overall effort estimate = productivity *size 



b) Bottom Up Method 

This is also known as "divide and conquer" technique. It is hierarchical decomposition of the test 
effort into stages, activities and tasks. 

• Planning 

Test environment & configuration 

• Test case creation 

• Test execution 

■ Again decompose the above activity in smaller packages which can be estimated in short 

period of time. 
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Estimate the total effort by understanding the duration and effort of each activity. 



c) Test Point Analysis(TPA) 

Test Point Analysis can be used to objectively prepare an estimate for black box testing 
(excluding performance testing). Test Case Point Analysis methodology is based on Test Case 
Points. Test Case Point is a Verification Point used to verify that the value on AUT matches with 
the expected value. This O/p value can be I/p data for other verification points. Following factors 
will have influence on number of Test Case Points: 

■ Complexity: It relates to the number of conditions in a function. More conditions almost 
mean more test cases and therefore a greater volume of testing work. 

■ Interfacing: The degree of interfacing of a function is determined by number of data sets 
maintained by a function and the number of other functions, which make use of those data sets. 

■ Uniformity: The extent to which the structure of a function allows it to be tested using 
existing or slightly modified specifications, i.e. the extent to which the information system 
contain similarly structured functions. 

Details about Test Case Point are as follows: 

■ Low Complexity Test Case Point: A Test Case Point having 1 to 3 Steps is considered as 
Low Test Case Point. 

■ Medium Complexity Test Case Point: A Test Case Point having 3 to 4 Steps is 
considered as Medium Test Case Point. 

■ Critical Complexity Test Case Point: A Test Case Point having 5 to 6 Steps is considered 
as Critical Test Case Point. 

Test Scripts can be defined in following three Complexity Levels: 



S.No. 


Level 


Definition 


1 


Critical 


If a Test Script is having 6 to 8 Test Case Points Or 
Verification Points. 


2 


Medium 


If a Test Script is having 4 to 5 Test Case Points Or 
Verification Points. 
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3 


Low 


If a Test Script is having 1 to 3 Test Case Points Or 






Verification Points. 



Breakdown between Testing Phases: 





Testing Phase 


% age 


Test 


Preparation 


10 


Scripting 


f 

(includes Functional Understanding) 




Specification 

(includes Test Conditions, Test Data 
identification, Test Script preparation) 


40 


Test 

Execution 


Test Script Execution and Defect 
Management 

(includes Smoke, System, Integration, End 
to End and Regression Test) 


45 


Completion 
(includes UAT) 


5 



Factors affecting Test Estimation 

Productivity Figure: It is based on knowledge and skill of test team members and is 
therefore specific to the individual organization. Productivity figure mentioned in these 
guidelines needs to be verified for couple of projects before implementation across GSI. 

Environmental Factor: Following environmental factors should be consider for test effort 
estimation: 

Test Tools: It reflects the extent to which testing is automated, or the extent to 
which automation tools are used for testing. 

Development Testing: It reflects the extent to which the development testing is 
down, a development test plan is available and test team is familiar with the actual test cases and 
test results 

Test Base: It reflects the quality of system documentation upon which the test 
under consideration is to be based. 
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Test Environment: It reflects the extent to which the test infrastructure in which 
the testing is to take place has previously been tried out. 

Testware: It reflects the extent to which the tests can be conducted using existing 

testware. 

Multiple Browsers: Effort estimation for testing on multiple browsers is more 
then testing on one browser. 



3.0 Conclusion 



The estimation technique guidelines explained in the earlier section can be enhanced to cover the 
various environmental factors. Pilot these guidelines for couple of projects in your organization 
and compare the estimated effort and actual effort. As we get proficient with its implementation 
we'll find that Estimated and Actual efforts are getting closer which will result in better 
execution of the testing projects. 



4.0 Definitions, Abbreviation and Acronyms 



Acronym 


Description 


QA 


Quality Assurance 


SEPG 


Software Engineering Process Group 


SMC 


Simple Medium Complex Method 


TPA 


Test Point Analysis 


FP 


Function Point 


UAT 


User Acceptance Testing 
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